Papers with internal reasoning processes
A Comparative Study of Faithfulness Metrics for Model Interpretability Methods (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods to reveal the reasoning processes of machine learning models are difficult to interpret due to their complexity. |
| Approach: | They propose to use diagnosticity and complexity to assess faithfulness of machine learning models . they propose to apply posthoc interpretation methods to reveal reasoning behind models based on internal reasoning . |
| Outcome: | The proposed interpretation metrics show conflicting preferences when comparing interpretations . sufficiency and comprehensiveness metrics have higher diagnosticity and lower complexity . |
Sycophancy Mitigation Through Reinforcement Learning with Uncertainty-Aware Adaptive Reasoning Trajectories (2025.emnlp-main)
Copied to clipboard
Mohammad Beigi, Ying Shen, Parshin Shojaee, Qifan Wang, Zichao Wang, Chandan K. Reddy, Ming Jin, Lifu Huang
| Challenge: | Existing studies show that large language models inadvertently foster sycophancy . scophancies are a tendency of models to blindly conform to user preferences without critical reasoning or self-reflection. |
| Approach: | They propose a method to reduce sycophancy by combining uncertainty-aware Monte Carlo tree search and progress-based reinforcement learning. |
| Outcome: | The proposed model outperforms baseline models in effectively reducing sycophancy while maintaining performance on out-of-distribution inputs. |
HAUNTATTACK: When Attack Follows Reasoning as a Shadow (2026.findings-acl)
Copied to clipboard
| Challenge: | Emerging Large Reasoning Models (LRMs) excel in mathematical and reasoning tasks, showcasing remarkable capabilities. |
| Approach: | They propose a framework that embeds harmful instructions into reasoning questions . they evaluate 11 LRMs and observe an average attack success rate of over 70% . |
| Outcome: | The proposed framework improves reasoning models by 13 percentage points over baseline. |
Tracing and Dissecting How LLMs Recall Factual Knowledge for Real World Questions (2025.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models have shown promising ability to perform commonsense reasoning. |
| Approach: | They propose a two-dimensional analysis framework that incorporates token back-tracing and token decoding to uncover how LLMs conduct factual knowledge recall. |
| Outcome: | The proposed framework shows that LLMs lack relevant knowledge but struggle to select the most accurate information based on context during the retrieval and rerank phase. |
Detecting Hallucinations in Retrieval-Augmented Generation via Semantic-level Internal Reasoning Graph (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for detecting faithfulness hallucinations are coarse or do not capture the models’ internal reasoning processes, making it difficult to learn. |
| Approach: | They propose a semantic-level internal reasoning graph-based method for detecting faithfulness hallucination using Large language models. |
| Outcome: | The proposed method achieves better overall performance compared to state-of-the-art baselines on RAGTruth and Dolly-15k. |